Adaptively Improving Long Distance Network Transfers with Logistics

Long distance data movement is an essential activity of modern computing. However, the congestion control mechanisms in the Internet’s Transmission Control Protocol (TCP) severely limit the bandwidth achieved by long distance data transfers. The throughput of such transfers can be improved by applying the logistical technique of breaking a single long distance transfer into multiple shorter transfers. This technique can result in signifi cantly improved throughput while still respecting the shared nature of the Internet by not attempting to circumvent the TCP congestion controls. This technique has been incorporated into an algorithm which attempts to dynamically schedule transfers for optimal throughput. The algorithm couples graph techniques with real-time latency and bandwidth measurements to discover the best path and adaptively respond to network dynamics. The algorithm shows improvements in speed and fl exibility over standard data transfer methods such as FTP. Specifi c transfers tests performed between Oak Ridge National Laboratory and a destination in Sunnyvale, CA show throughput increases by a factor of two.